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Abstract 

Recently, many practical algorithms have been proposed to recover 
the sparse signal from fe'wer measurements. Orthogonal matching pur- 
suit (OMP) is one of the most effective algorithm. In this paper, we 
use the restricted isometry property to analysis the algorithm. We 
sho'w that, under certain conditions based on the restricted isometry 
property and the signals, OMP -will recover the support of the sparse 
signal ■when measurements are corrupted by additive noise. 



1 Introduction 

Compressed sensing sho"ws that it is high possibility to reconstruct sparse 
signals from their projection onto a small number of random vectors, pos- 
sibly corrupted by noise. Let ||x||o denote the number of nonzero entries of 
vector X. If ||x||o < K, a signal x is said to be K-sparse. Let ^ be an m x n 
measurement matrix 'with m < n. In compressed sensing, ■we are interested 
in recovering the -fC-sparse signal x from 

y = Ax + z, (1.1) 
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where z is the noise term. Then, the approach would be to solve the following 
/q minimization problem: 

min||Ax — y||2 subject to ||x||o < -ftT. (1-2) 

X 

A greedy algorithm named orthogonal matching pursuit (OMP) is one of the 
efficient approach to solve (|1.2p . The basic idea of this iterative algorithm is 
to find the support of the unknown signal. At each iteration, one column of 
A that is the most correlated with the residue is selected. Then the residue 
is updated by projecting y onto the linear subspace spanned by the columns 
that have been selected. Basic reference for this method are [3 [T3| and 
|16j . There are several natural stopping criteria for OMP [17] . Let be 
the residual in the each iteration. 

(1) Halt after a fixed number of iterations: k = K. 

(2) I2 bounded noise: Halt when no column explains a significant amount 
of energy in the residual: ||rfc||2 < £■ 

(3) loo bounded noise: Halt when no column explains a significant amount 
of energy in the residual: || A*rfc ||oo < e where A* denotes the transpose 
of ^. 

The mutual incoherence property [8] and the restricted isometry property 
[5] of the measurement matrix have been used for the analysis of OMP. Let 
Ai be the ith column of the matrix A. In this paper we assume ||^i||2 = 1; 
i = 1, . . . n. The mutual incoherence is defined by 

^l{A) = max\{Ai,AJ)\, 

A given matrix A satisfies the restricted isometry property of order K if 
there exist a 5k such that 

(l-(5i^)||x||^ < px||^ < (l + 5/^)||x||^ for ah ||x||o < i^. (L3) 

The smallest constant 6k is called the restricted isometry constant. Many 
types of random matrices satisfy the RIP with high probability, such as 
subgaussian random matrix and random partial Fourier matrix [15] . The 
mutual incoherence property is stronger than the RIP: 5k+i < Kn{A). 

In [16], Tropp has shown fJ-{A) < 2K-1 is a sufficient condition for re- 
constructing any X-sparse signal in the noiseless. Then Cai, Wang and Xu 
proved this condition is sharp in [2]. In [6], Davenport and Wakin have 
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showed that there exist matrices satisfying some RIP but not the mutual 
coherence condition via numerical experiments. This motivated them to 
establish the RIP-based sufficient conditions. They have proved that the 
restricted isometry constant S3K < is sufficient for OMP to recover 

any JC-sparse signal in K steps. Several papers have improved the sufficient 
condition, such as [TO] and [11,]. Very recently, Mo and Shen have improved 
the sufficient condition to 

For any K > 2, they also constructed a matrix with the restricted isometry 
constant Sk+i = such that OMP can not recover some X-sparse signal 
X in X iterations. Hence, the estimate ()1.4p is near-optimal. 

For the noise case, Cai and Wang have provided coherence-based guaran- 
tees for OMP 1^. This subject was also considered in [9] and [18J. However, 
there are few results on the general model (jl.ip by using the RIP. Follow- 
ing the line of |12j . we investigate the OMP in the noise case under the 
RIP-based conditions. 

The rest of paper is organized as follows. In section [2l we shall introduce 
some notations and investigate some properties of the restricted isometry 
constants. In section 3, the main results are established for OMP recovering 
the sparse signals with noise. 



2 Preliminaries 

Before going further, we introduce some notations. Suppose T is a subset 
of {1, . . . , n}. Let = {1, 2, . . . , n} \ T. For a given matrix A, denote 



At 



i £ T, 
otherwise. 



For convenience, Aj- also denotes the submatrix of A corresponding to T. 
We use the same way to define for the vector x G R". Thus, we have 

At^ = Axt = At^t- 

The pseudo inverse of a tall, full-rank matrix A is defined by A"^ = {A* A)~^ A. 
The support of x = {xi, . . . , Xn} is denoted by supp{x) = {i : Xi ^ 0}. Let 
Cj be the ith coordinate unit vector in R". We denote 

S'i(x) := (^ei,^x), i = l,...,n. 
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Table 1: Orthogonal Matching Pursuit 



Input: A, y 

Set: = 0, xo = 0, A; = 1 
while not converge 

rfc = y - An^_^Xk-i 

rifc = fife-i U argmaxi |(^ei,rfc)| 

k = k + l 
end while 

set: icQ^ = x^, x^c = 
Return: x 



and 



S't(x) := max |S'i(x)|, 



E{z) := max \{Aei,z)\. 

ie{l,...,n} 



Table [T] shows the framework of OMP. 

Now we investigate some properties of the restricted isometry constant. 
Lemma l2.ll were established by Needell and Tropp in [13j. 

Lemma 2.1. Let x be a K-sparse vector. Suppose the matrix A has the 
restricted isometry constant 6k- Then for T C supp{x), 

1. \\A^At<=x\\2 < 5_ft:||xTc ||2- 

2. \\{A*.,ATr^xh<j^\\xh. 

The following lemma was obtained by Cai and Wang in [3]. 

Lemma 2.2. Let x he a K-sparse vector with 17 = supp{x). Suppose that 
the matrix A has the restricted isometry constant 6k- Then for T C 

1. il-6K)\\xn\Th < \\A*^\Ti^-ATA\.)An\T^n\Th < {l + 6K)\\^n\Th- 

2. (1 - <5i^+l)||xTc||2 < \\AiI - 4^)x||2. 

Lemma 2.3. Let x be a K-sparse vector. Suppose that the matrix A has 
the restricted isometry constant 6k- Then for any T C supp{x) 

||(/-4^)x||2<|^. (2.1) 
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Proof. Split X = xx + xj-c , we have 

(/ - aI.A)x = XT + XTc - aI^Atxt - AlpArcXT'^ 

= XT + X^c — XT — A^^TcXj-c 
= XT': — AtpAx'^XT'^ ■ 

By Lemma l2.ll we get 

||(/-^^^)x||2 < \\xtc\\2+\\A\.AtcXt42 

< \\xtc\\2+\\{AtAt)~^AtAt<:Xtc\\2 

ii ii ii ii 

< ||xtc||2 + - — ||xtc||2 

I - Ok 

- I -6k' 



□ 



3 I2 Bounded Noise 

In this section, we shah prove the main results of the paper. Both the stop- 
ping rule 2 and the stopping rule 3 of OMP for the noise case are considered. 
We first consider the noise z is bounded by ||z||2 < B2- Then the stopping 
rule is Hr^lb < B2. 

Lemma 3.1. Suppose 6k+i < have 

{1 - 6K+if - Sk+i{1 + ^) > 0. (3.1) 
Proof. Simple calculate shows that (j3.ip is equal to 



+ Sk+1 > + 3. 

OK+1 

Thus, 5k+i < rh „ is the stronger condition. □ 
The following results is a key tool in this paper. 

Theorem 3.2. Assume 5k+i < "y^ljl^- ^^''^ ^''^V given K -sparse signal x. 
Suppose that the measurement matrix A has the restricted isometry constant 
5k+i satisfying 



I II 2{l - 5K+i)E{^k)VK^ .3 2^ 

{1-5k+i?-5k+i{1 + VK^) 
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where = (/ — Aq^,aI^^)z. Then OMP selects an index of the support of:^ 
at the {k + l)th iteration. 

Proof. For a given X-sparse signal x, denote the support of x by $7. Consider 
the {k + l)-th iteration, 

= A^ + z-An,Al^{A^ + z) 

= Ml-AlA)^ + {I-An,Al)z. 



For simphfy, let tfc = (/ — A^^A)x. Then we get 

{Aei,rk+i) = (y4ej,y - An^Xfc) 
= {Aei,Atk + Zk) 
= Si{tk) + {Aei,Zk). 

Note that the residual are orthogonal to all the selected columns of A, 
so no index is selected twice. Thus, the sufficient condition for choosing an 
index from $7 \ $7^ in the {k + l)th iteration is 

Sn\n,itk) - E{zk) > \Si{tk)\ + E{zk) for all iGQ". (3.3) 

In the rest of the proof, we shall give a sufficient condition for (j3.3|) holds. 
Note the support of is a subset of fi. By Lemma 2.1 in [4], we have 

\Si{tk)\ = \{Aei,Atk)\ < 6K+i\\tk\\2 for ah i G Q". (3.4) 

Combine (l2T]) and (lOl) leads to 



\Si{tk)\ < , W^nih, for all i G Q". (3.5) 
By Lemma 12.21 we obtain 

ll^mo, ^**:I|2 (1 — '^_ft:)||xn<: II2 

5„,„.fa)>^^> ' (3.6) 

It follows from ()3.5p and ()3.6p that the sufficient condition for ()3.3p holds is 
{l-6K)\\^nih Sk+1 II II ^.p. ^ 

, Xoc 2 > ZlSIZi. 1 



which is simplified to (13. 2p . □ 
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Theorem 3.3. Suppose llzlU < B2 and 5k+i < -y^ — • Then OMP with the 

V K+3 

stopping rule ||rfc||2 < B2 finds the support ofx if all the nonzero coefficients 
Xi satisfy 

(1 - - + VK) 

Proof. We first estimate the E(zk). Since ||z||2 < B2, we have 

\{Aei,Zk)\ < \\Ai\\2\\{I - An,Aljz\\2 < ||z||2 < S2. 

This implies 

E{zk) < B2. 



Hence, (j3.2p is followed by 



' (l-S,(+l)2-%+,(l + Vlf^) ^ ' 



Since 



> mm 



3.8p is implied by 

2(1 - ^^+i)i32 ^ forahiG^^ 



(l-(5i^+i)2-,5K+i(l + V^)' 



Now we prove that the OMP do not stop for some j + 1 < k. Consider 
the j + 1 iteration for some j + 1 < fc, by Lemma [221 we have 

||rj+l||2 = ||y - ^f7^Xj||2 

= P(/-4^yl)x + (I-An,4pz||2 

> P(/-4^yl)x||2-||(I-An,4^,)z||2 

> (l-5K+l)||xf^c||2-S2. (3.9) 

By ()3.7p . we obtain 



It follows from and (f3T0]l that ||rj+i||2 > B2. □ 
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We assume that z is zero-mean white Gaussian noise with covariance 
^"^Imxm- Cai, Xu and Zhang have show that z ~ N(0,(j'^lmxm) satisfies 

P{z G ^2) > 1 - 1/m 

where B2 = |z:||2:||2< ay/ni + 2-v/mlogm|. With this argument and 
Theorem 13.31 we obtain the fohowing result. 

Theorem 3.4. Suppose z ~ N{0,a'^lmxm), ^K+i < and nonzero 

coefficients Xi satisfy 



2(1 — 5/^+1 )(T a/ m + 2\Jm\o'gm 
^ (1 - - ,5,^+1(1 + • 



Then OMP with the stopping rule \\rk\\2 < cr\/ m + 2-v/m logm ^nds f/ie 
support of X with probability at least 1 — 1/m. 

Now we give the RIP-based sufficient conditions for OMP with /qo bounded 
noise case. Then the stopping rule is ||^*rfc||oo < Boo- 

Theorem 3.5. Suppose ||j4*z||oo < Boo and 6k+i < ■ Then OMP 

with the stopping rule \\A*r\\oo < B^o finds the support ofx if all the nonzero 
coefficients Xi satisfy 

{\-Sk+i?-Sk+i{\ + -/K)\ 

Proof. Since the proof is similar as the proof of Theorem 4 in [3j . We include 
a sketch for the completeness. To make sure (|3.2p of Theorem 13.21 hold, we 
first give an estimation of E{zk) in the {k + l)-th iteration. We have 



|(^ei,zfc)| < \A*x\ + \{Aei,An,Al^i)\ <Boo(^ + -^ 



Together with (j3.1ip . it implies that p.2p holds. Now consider the t-th 
iteration with t < k + 1. We obtain 



> ^00 • 
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The second inequality is implied by (|3.1ip . Therefore, the OMP does not 
stop after t-th. iteration. 

□ 
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